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(54) Image sensing apparatus arul image processing method therefore 



(57) An image sensing apparatus comprising: a 
CCD. optical system for forming an image on the CCD; 
A/D converter for converting an image signal oulputted 
by the CCD to a dicptal signal; color interpolator for per- 
forming color interpolation on the digital signal con- 
verted by the A/D converter and generating image data 
on R. G and B color planes; color space converter for 
converting RGB color space to YUV colorimetric sys- 



tem; and median filter for reducing pseudo color compo- 
nents, generated by the color interpolator, by controlling 
color difference signals U stnd By cutting high fre- 
quency components of the signal, filtered by the median 
filter, then thinning out YUV signals and performing 
JPEG conpression. pseiKlo color conponents gener- 
ated by color interpolation can be reduced. 
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Description 

BACKGROUND OF THE INVENTION 

5 [0001 ] The present invention relates to an image sensing apparatus such as a digital camera or digital video cam- 
era or the like, and an image processing method for the image sensing apparatus. 

[0002] A conventional singleiDlate color cOgital camera Is a lens-integrated system .and Is generally incapable of 
exchanging a lens. For this reason, an optical low pass filter and infrared ray (IR) filter or the like are provided before a 
CCD. which generates image signals according to light passing through the lens, thereby achieving a certain degree of 

10 effectiveness in reducing moire or pseudo colors (erroneous colors). However, in a case of a camera which allows lens 
exchange, an IR filter can be provided before a CCD by forming a thin film on a glass surface of the CCD. However, 
securing space for an optical low pass filter before a CCD causes to increase the size of the camera body. Furthermore! 
although such optical low pass filter can reduce moir6 or pseudo colors to a certain degree, a problem arises in that 
spatial frequency of an image decreases, and as a result, the image loses exact-focused characteristic such as that 

15 obtained in silver chloride photographs. Because of the above reason, the importance of an optical system without an 
optical low pass filter is increasing. 

[0003] Furthermore, even if such optical low pass filter is provided, in a case of a digital camera employing a single- 
plate CCD. pseudo colors are still generated when color interpolation is performed. As exerrplif ied by the Bayer pattern, 
when a number of R (red) and B (blue) color components is less than that of G (green) color components, the gap 

20 between these pixels becomes large, and this causes generation of pseudo cofors in color interpolation. Furthermore, 
a method of generating three cotor planes with a digital filter or the like is available as a conventional color interpolation! 
However, since the order of the digital filter is limited, original image data resolution cannot be sufficiently expressed. 
[0004] In view of the above, conventionally proposed is to perform image processing, particularly color interpola- 
tion, disclosed in US Patent No. 5.373.322 or No. 5.629.734, to obtain high resolution image data. This technique is 

25 effective to reduce pseudo colors, but not effective for moire . 

[0005] Even with the conventional technique, isolated pseudo colors, generated particularly around small charac- 
ters or the like, cannot completely be eliminated. In view of this, it is proposed to convert color space, e.g.. from RGB 
to L*a*b*, by an application program operating In a computer, and perform processing such as filtering on a* and b* to 
remove pseudo colors. 

30 [0006] However, even if such conventional technique is employed, pseudo colors in cotor interpolation cannot com- 
pletely be eliminated. This is further described below. 

[0007] As shown in Fig. 19, if vertical white lines are exposed to green (G) and red (R) lines of a CCD at pixel pitch 
and conventional color interpolation is performed, red (R) and yellow (Y) vertical stripes are obtained (in Fig. 19, hatch- 
ing portion indicates black where data is 0). Similariy. as shown in Rg. 20, if vertical white lines are exposed to green 
(G) and blue (B) lines of a CCD at pixel pitch and color interpolatfon is performed, blue (B) and cyan (C) vertical stripes 
are obtained. Note that even if the stripes in Rgs. 1 9 and 20 are horizontal instead of vertical, the same resufts. i e red 
(R) and yellow (Y) stripes or blue (B) and cyan (C) stripes, are obtained. 

[0008] Furthermore, as another pattern, if white pixels having a checker flag pattern shown in Fig. 21 are exposed 
to R and B of a CCD and color interpolation is performed, a magenta (M) image is obtained despite the original white 
color. Still further, if white pixels having a checker flag pattern shown in Fig. 22 are exposed to G of the CCD and color 
interpolation is performed, a green (G) image is obtained despite the original white color. 

[0009] In order to remove such pseudo colors, the color space is converted from RGB to. for instance. L*a*b* and 
filtering is performed on each of the a* and b* color spaces by an application program operating in a personal computer. 
However, in the case of checker flag patterns shown in Figs. 21 and 22. a low frequency image of green cannot be dis- 
tinguished from that of magenta, making it impossible to determine pseudo colors. If the frequency band of color differ- 
ence is forcefully limited for JPEG conpression. a dull (blurred) image is obtained. As a result, although the level of 
pseudo colors is reduced, the pseudo color components are spread out to the peripheral pixels, causing problems. 
[0010] Furthermore, if the image data is conpressed according to JPEG or the like without performing above- 
described pseudo color removing processing, block noise or the like may be caused. Moreover, if the frequency band 
of color difference is forcefully limited for JPEG compression, an image becomes blurred. As a result, although the level 
of pseudo colors is reduced, the pseudo color components are spread out to the peripheral pixels, causing problems. 
In order to eliminate such pseudo colors, it is Ideal to employ a three-plate camera using three image sensing devices 
for each color component. However, such construction increases the size and cost of the camera. 

55 SUMMARY OF THE INVENTION 

[00111 The present invention is made in consideration of tiie above situation, and has as a concern to provide an 
image sensing apparatus capable of reducing pseudo cotors generated by color interpolation, and an image processing 
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method for the image sensing apparatus. 

[001 2] Furthermore, another concern of the present invention is to provide an image sensing apparatus which can 
reduce pseudo colors without increasing the size or cost of the image sensing apparatus, and an image processing 
method for the image sensing apparatus. 
5 [001 3] Furthermore, another concern of the present invention is to provide an image sensing apparatus which can 
reduce pseudo colors with r^atively a simple construction, and an image processing method for the image sensing 
apparatus. 

[0014] Furthermore. Another concern of the present invention is to provide an image sensing apparatus which can 
reduce pseudo colors in isolated pixels with relatively a simple construction, and an image processing method for the 

10 image sensing apparatus. 

[0015] An image sensing apparatus according to the present invention can have the following configuration. 
[0016] One aspect of the present invention provides an image sensing apparatus conrprising: an image sensing 
device; image forming means for forming an image on said image sensing device; A/D conversion means for converting 
an image signal outputted by said image sensing device into a digital signal; color interpolation means for performing 

IS color interpolation on the (£gitai signal converted by said A/D conversion means and generating image data on a plu- 
rality of color planes; color space conversion means for converting a color space of the plurality of color planes to a color 
space of another colorimetric system; and isolated point removing means for reducing a pseudo color, generated by 
said color interpolation means, by controlling a color difference signal converted by said color space conversion means. 
[001 7] Another aspect of the present invention provides an image sensing apparatus comprising: an image sensing 

20 device; image forming means for forming an Image on said image sensing device; A/D conversion means for converting 
an image signal outputted by said image sensing device into a digital signal; color interpolation means for perfornting 
color interpolation on the digital signal converted by said A/D conversion means and generating image data on a plu-- 
ralrty of color planes; pseudo color removing means for reducing a pseudo color component included in image data, on 
which color interpolation is performed by said color interpolation means; color space conversion means for converting 

25 a color space of the plurality of color planes, where a pseudo color is reduced by said pseudo color removing means, 
to a color space of another colorimetric system; and compression means for compressing image data where color 
space is converted by said color space conversion means. 

[0018] Herein, the isolated point removing means includes an isolated point-removing filter which replaces a value 

of a pixel of interest with a substantial median pixel value of peripheral pixels of the pixel of interest Also, the isolated 
30 point removing filter includes a median value filter or median filter. 

[0019] Moreover, the color interpolation means generates image data in R, G and B planes. 

[0020] Further, the color space conversion means may convert RGB color space to YUV color space. 

[0021] Still further, the color space conversion means may convert RGB color space to Y. R-Y. B-Y color space. 

[0022] Still further, the color space conversion means may convert RGB color space to G. R-G. B-G color space. 
35 [0023] Moreover, the image forming means includes an infrared ray filter, or an infrared ray filter and optical low 

pass filter. 

[0024] Furthermore, the image processing method for the image sensing apparatus according to the present inven- 
tion has the following steps. 

[0025] A further aspect of the present invention provides an image processing method for an image sensing appa- 
40 ratus which includes an image sensing device euxj generates an image signal corresponding to an image formed on the 
image sensing device, comprising: an A/D conversion step of converting an image signal outputted by the image sens- 
ing device into a digital signal; a color interpolation step of performing color interpolation on the digital signal converted 
in said A/D conversion step and generating image data on a plurality of color planes; a separation step of separating 
the image data in the plurality of color planes into luminance data and color difference data; an extraction step of 
45 extracting a high frequency component from the luminance data separated in said separation step; and a pseudo color 
removing step of reducing a pseudo color generated in said color interpolation step, in accordance with the high fre- 
quency component of the luminance data extracted in said extraction step and hue data obtained from the color differ- 
ence data. 

[0026] Herein, the pseudo color removing step comprises: a determination step of determining whether or not the 
so high frequency component of the luminance data and the hue data fall within a color range of the pseudo color; and a 
step of reducing a value of the color difference data, determined to be within the color range in said determination step. 
[0027] Moreover, in the color space conversion step, it is preferable that the color space be converted to YUV, or Y. 
R-Y B-Y. or G, R-G. B-G color space. 

[0028] Moreover, the image processing method may further comprise a step of removing a high frequency compo- 
55 nent from a color signal, from which the pseudo color is removed in said pseudo color removing step. 

[0029] Herein, the color range of the pseudo color includes a color area from red to yellow, and a color area from 
blue to cyan. 

[0030] Other features and advarrtages of the present invention will be apparent from the following description taken 
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in conjunction with the accompanying drawings. In which like reference characters designate the same or similar parts 
throughout the figures thereof. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0031] TTie accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate 
embodiments of the invention, and together with the description, serve to explain the principles of the invention. 

Fig. 1 is a block diagram showing, mainly, a construction of an image processing unit of a digital camera according 
to the first embodiment of the present invention; 

Figs. 2A and 2B are conceptual views explaining pseudo color removal according to embodiments of the present 
invention; 

Fig. 3 is a block diagram showing, mainly, a construction of an image processing unit of a digital camera according 
to tiie second embodiment of the present invention; 

Fig. 4 is a block diagram showing, mainly, a construction of an image processing unit of a digital camera according 
to tiie third embodiment of the present invention; 

Fig. 5 is a block diagram showing, mainly, a construction of an image processing unit of a digital camera according 
to the fourth embodiment of the present invention; 

Rg. 6 is a block diagram showing, mainly, a construction of an image processing unit of a digital camera according 
to the fifth embodiment of the present invention: 

Rg. 7 is a block diagram showing a configuration of a pseudo color removing processor according to the fifth 
embodiment; 

Rg. 8 is a block diagram showing a configuration after the gamma converter shown in Fig. 6, for a digital camera 
according to the sixth embodiment of the present invention; 

Rg. 9 is a block diagram showing, mainly, a construction of an image processing unit of a digital camera according 
to the seventh embodiment of the present invention; 

Rg. 10 is a conceptual view for explaining color interpolation according to embodiments of tiie present invention; 
Rg. 1 1 is an explanatory diagram showing a construction of a high frequency filter according to embodiments of 
the present invention; 

Rg. 12 is a graph explaining processing of a pseudo color suppression unit according to embodiments of the 
present invention ; 

Rg, 13 shows a hue range of pseudo colors in pseudo color suppression processing according to embodiments of 
the present invention; 

Rg. 14 is a flowchart explaining pseudo color removing processing according to the seventh and eighth embodi- 
ments of the present invention; 

Rg. 1 5 is a block diagram showing, mainly, a construction of an image processing unit of a digital camera accoreiing 
to the eighth embodiment of the present invention; 

Rg. 1 6 is a block diagram showing a construction of an image processing unit of a digital camera according to the 
ninth embodiment of the present invention; 

Fig. 1 7 shows a hue range of pseudo colors in pseudo color suppression processing according to the ninth embod- 
iment of the present invention; 

Fig. 18 is a t}lock diagram showing a construction of an image processing unit of a digital camera according to the 
tenth embodiment of the present invention; 

Rg. 19 is a conceptual view explaining a general pseudo color generation process; 
Rg. 20 is a conceptual view explaining a general pseudo color generation process; 
Fig. 21 is a conceptual view explaining a general pseudo cobr generation process; and 
Rg. 22 is a conceptual view explaining a general pseudo color generation process. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0032] Preferred embodiments of the present invention will be described in detail in accordance with the acconpa* 
nying drawings. 

[First Embodiment] 

[0033] Rg. 1 is a block diagram showing, mainly, a construction off an image processing unit of a digital camera 
according to a first embodiment of the present invention. 

[0034] Light 1 . incident upon the digital camera according to the first embodiment, passes through lens 2. then the 
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amount of the light is adjusted by a diaphragm 3. and an image sensing device 5 (hereinafter referred to as a CCD), 
e.g.. CCD or CMOS, is exposed while a shutter (not shown) is open. Before the light 1 is incident upon the CCD 5. an 
optical low pass f Oter 20 limits a spatial frequency of the light 1 to reduce moir4 in a way that an optical area of a long 
wavelength is cut off by an infrared ray (IR) filter 4 so that the CCD 5 does not detect light in the infrared region. By the 

5 light incident up>on the CCD 5. the amount of electric charge corresponding to the intensity of light is accumulated in the 
CCD 5. The amount of electric charge is amplified by a predetermined gain by a CDS • AGC 6. and converted to digital 
data by an A/D converter 7. Image data converted to digital data in the foregoing manner is transferred to a white t>al- 
ance circuit 8 where R. G and B gains are adjusted. Then, by a color Interpolator 9. the image data is generated in. for 
instance, three color (R, G and B) planes. The image data expressed in the R. G and B planes are then transferred to 

w a masWng processor 10 where adjustment is made for the hue of the R, G and B colors. Then, a gamma converter 1 1 
performs necessary processing for displaying an image on a display or the like. 

[0035] Next, compression according to JPEG or the like is performed on the image data sirKe the data in the R, G 
and B color planes has a large arhount of data. 

[0036] Herein, first, an RGB/YUV converter 12 converts the image data from R, G and B signals to Y color differ- 
15 ence signals. For instance, R. G and B data are respectively converted to Y. U and V as follows: 

Y» 0.29900 X R + 0.58700 xG + 0.11400 xB (1) 

U= -0.16874 X R - 0.33126 x G + 0.50000 x B (2) 

20 

V=r 0.50000 X R . 0.41869 x G - 0.08131 x B (3) 

[0037] Next, with respect to U and V. each representing a color difference signal, a pseudo color generated by color 
interpolation is reduced by median filters 13a and 13b. Low pass filters (LPF) 14a and 14b remove high frequency com- 
25 ponents to limit the band. Then, a thinning circuit 1 5 performs thinning processing which complies with a predetermined 
compression method, e.g., from Y:U:V = 4:4:4 to Y:U:V = 4:2:2 or Y:U:V = 4:1 :1. Then, a compression circuit 16 adopt- 
ing JPEG or the like performs conrpression. 

[0038] Figs. 2A and 2B explain the principle of processir^ performed by the median filters 13a and 13b according 
to the first (and subsequent) embodiment. Herein. Fig. 2A shows 3x3 pixels peripheral to a pi^ of interest, and Rg. 

30 2B shows a graph of pixel values arranged in the ascending order. 

[0039] Herein, for instance, 3x3 pixels per^^heral to a pixel of interest a*22 are extracted, and the pixel values 
(color difference values) are arranged in the ascending order. Then, the value "20" of the pixel of intisrest is replaced 
with the value "10** of a'^3. which is the substantial median value of the nine pixels. Such processing is performed by 
the median filters 1 3a and 1 3b. serving as isolated point removing filters. By this, isolated pseudo colors can be reduced 

35 without largely decreasing resolution. Note that since the configuration of the median filter is well known (e.g., disclosed 
in Japanese Patent Application Laid-Open Na 5-233804 or No. 5-12437), detailed descrqstion thereof will be omitted. 
[0040] As has been described above, according to the first enrtedimerift, a color space of an inputted image signal 
is converted from RGB to YUV color space, and an isolated pixel is removed from color difference signals U and V of. 
the YUV signal by the median filter or the like. By virtue of this, pseudo colors generated by color interpolation can be 

40 reduced. 

[Second Embodiment] 

[0041] In the above-described first embodiment, color conversion from RGB to YUV color space is performed and 
45 pseudo color reduction processing is performed in the YUV color space before image compression is performed 
according to JPEG or the like. However, pseudo cok>r reduction processing is not necessarily performed in YUV color 
space, but may be performed in another color space. 

[0042] According to the second embodiment of the present invention, pseudo color removal is performed by a 
medieui filter, after a color space of input image data is converted from RGB color space to Y, R-Y, B-Y color space. 

so [0043] Fig. 3 is a k>lock diagram showing a construction of an image processing unit of a digital camera according 
to the second embodiment. For the components that are common to the construction shown in Rg. 1, the same refer- 
ence numerals are assigned and description thereof will be omitted. The second embodiment largely differs from the 
foregoing first embodiment in that the second embodiment does not necessitate JPEG compression and that the YUV 
conversion can be eliminated since the pseudo-color-removed image data is stored in a memory 19. Therefore, the 

55 construction of the image processing unit can be made simple, which contributes to the effect of the second emtxxii- 
ment This is also applicat)le to a case of performing compression according to JPEG or the like. 
[0044] According to the second embodiment, in order to achieve reduction in the amount of calculation compared 
to the case of converting data from RGB to YUV color space, image data is converted from RGB color space to Y R-Y, 
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B-Y color space by. for instance, using a Y R-Y B-Y converter 1 7a. 

[0045] Herein, for instance, color space conversion is executed in the following manner: 

Y = 0.29900 X R + 0.58700 x G + 0.11400 x B (4) 

R-Y = R-Y (5) 

B-Y = B-Y (6) 

[00461 Then, the median filters 1 7b and 1 7c respectively executes median filtering of the R-Y signal and B-Y signal. 
Herein, the equation can be further simplified as follows to obtain luminance signal Y, color difference signals R-Y and 
B-Y: 

Y = R + 2xG + B=Ye (7) 

R-Y = R-Ye (8) 

B-Y = B-Ye (9) 

[0047] In this case. R-Ye and B-Ye are calculated with a luminance signal Ye which is a simplified (approximate) 
form of the luminance signal Y By executing median filtering respectively to R-Ye and B-Ye, pseudo colors can be 
removed. Although the resultant values are slightly different from values of Y R-Y and B-Y. a similar effect is achieved 
and calculation can be performed easily. Next, an RGB converter 17d respectively converts the Y. R-Y and B-Y signals 
back to R. G and B signals. The image data represented by R. G and B signals is subjected to masking by a masking 
processor 1 0 and to a gamma conversion by a gamma converter 11 . and then stored in the non-volatile memory 19. 

[Thiid Embodiment] 

[0048] In the above-described second embodiment, the luminance signal Y Is consciously calculated from R. G and 
30 B signals. However, in image processing. G signal of the R. G and B may substitute for the luminance signal Y * 

[0049] The third embodiment is characterized by emplc^ng G. R-G and B-G signals instead of Y. R-Y and B-Y sig- 
nals employed in the second embodiment, and executing median filtering to R-G and B-G signals respectively. 
[0050] Fig. 4 is a block diagram showing a construction of an image processing unit of a digital camera accofding 
to the third embodiment of the present invention. For the components that are common to the construction shown in 
35 Fig. 3, the same reference numerals are assigned and description thereof will be omitted. 

[0051] According to the third embodiment, the amount of calculation is reduced further when conpared against the 
case of converting R. G and B signals to luminance signal Y and color difference signals R-Y and B-Y The G. R-G. B- 
G converter 18a generates G, R-G and B-G signals from R. G and B signals using the following equations: 
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Y = G (10) 
R-Y = R-G (11, 
B-Y = B-G (12) 



[0052] The median filters 1 8b and 18c respectively execute filtering to the R-G and B-G signals, thereby removing 
pseudo colors generated by color interpolation. Then, the RGB converter 18d converts the G. R-G and B-G signals 
back to RGB signals, which are then subjected to the masking processor 1 0 and gamma converter 1 1 . and then stored 
in the non-volatile memory 1 9. 

[0053] As has been described al>ove. since the third embodiment employs G signal in place of luminance signal Y, 
pseudo color reduction can be realized with a less amount of calculation, although there is a possibility that luminance 
components of the RGB signals may be influenced by the result of median filtering. 

[Fourth Embodiment] 

[0054] The above-described first to third embodiments describe the case of employing an optical low pass filter 

before the CCD 5. Note that the configuration before the CCD 5 is not shown In Figs. 3 and 4. 

[00551 1>^e fourth embodiment describes a case where no problem arises even though the optical system does not 
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comprise such optical low pass filter. 

[0056] Fig. 5 is a block diagram showing a construction of an image processing unit of a digital camera according 
to the fourth embodiment of the present invention. For the components that are common to the construction shown in 
Fig. 1. the same reference numerals are assigned and description thereof will be omitted. The fourth emtxxJiment is 
5 . characterized in that the IR filter 4 for cutting infrared rays is provided before the CCD 5, arxi that the optical low pass 
fitter 20 employed in Fig. 1 is renxived. The configuration after the CCD 5 is the same as that shown in block diagram 
of Rg. 1. 

[0057] However, when the optical low pass filter 20 is removed, pseudo colors are generated more than the case 
of including the optical low pass filter 20. In view of this, the number of elements in median filters 130a and 130b in the 

10 fourth embodiment is increased from the number of elements of the median filters according to first to third embod- 
iments. By this, the median filtering can be executed in a wider area, and pseudo colors can be reduced. 
[0058] Accordingly, since the fourth embodiment is realized by simply mounting tiie IR filter 4 on the glass surface 
of the CCD 5, the apparatus main kxxiy can be nnade small and cost reduction can be achieved. In addition, image sig* 
nals of high resolution can be obtained. 

IS [0059] For the isolated point removing filter, filtering called smoothing (Edge Preserving Smoothing), which pre- 
serves edges, may be employed besides the above-described median filter. More specifically, a window W(i, j) is set 
with a pixel f(x. y) as its center, and an output g(x. y) is calculated. Assuming that the window size is m x n, the following 
equation stands: 

20 m n 

9(Ky) = 2:2:^^?^ f(x-hyi) 



25 For instance, assuming a matrix M in which data "1 " is arranged in 3 x 3 matrix, the following equation stands: 

W = M/9 

[0060] For an alternative isolated point removing filter, another smoothing (Maximum Homogeneity Smoothing) 
30 which preserves edges may be employed. This snx>othing enables selection of a most uniform area in the neighd3or- 
hood of a pixel. When the weight coeff ident R is defined as follows: 

weight coefficient R ~ {min (at)} / at where an average value of each block is ^i. distritxition is ai, and m (arbitrary). 
35 the oulput g(x, y) is obtained in the following manner: 

8 8 

C 40 

[0061] Besides the above, filtering called hysteresis smoothing may be employed. More specifically, a hysteresis 
characteristic is produced for a data value, and noise is absoibed by the hysteresis characteristic. Any of the aforemen- 
tioned filters may be used as long as interpolation or smoothing is performed by using outputs of peripheral pixels of an 
45 isolated point having a pixel value largely different from peripheral pixel values. 

[Fifth Embodiment] 

[0062] Fig. 6 is a block diagram showing, mainly, a construction of an image processing unit of a digital camera 
50 according to the fifth embodiment of the present invention. For the components that are common to the foregoing 
embodiments, the same reference numerals are assigned. 

[0063] Light 1, incident upon the digital camera according to the fifth enr^odiment. passes through the lens 2. then 
the amount of the light is adjusted by the diaphragm 3. and the image sensing device 5 (hereinafter referred to as a 
CCD), e.g., CCD or CMOS, is exposed while a shutter (not shown) is open. Before the light 1 rs incident upon the CCD 
55 5. a frequency region of a long wavelength is cut off by the IR filter 4 so that the CCD 5 does not detect light in the infra- 
red region. By the light incident upon the CCD 5. the amount of electric charge corresponding to the intensity of ligtrt is 
accumulated in the CCD 5. The amount of electric charge is amplified by a predetermined gain by the CDS • AGC 6, 
and converted to digital data by the A/D converter 7. Image data converted to digital data in the foregoing manner is 
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transferred to the white balance circuit 8 where R, G and B gains are adjusted. Then, by the color interpolator 9. the 
image data is generated in. for instance, three color (R. G and B) planes. The image data expressed in the R. G and B 
planes are then transferred to the masking processor 1 0 where adjustment is made for the hue of the R. Q and B colors. 
Then, the gamma converter 1 1 performs necessary processing for displaying an image on a display or the like. 
[0064] Next, a pseudo color removing processor 21 reduces pseudo colors generated by color interpolation of the 
color interpolator 9. Further, compression processing according to JPEG or the like is performed on the image data 
since the data in the R. G and B color planes has a large amount of data. Herein, first, the RGBA^UV converter 12 con- 
verts the image data from R. Q and B signals to Y color difference signal, and with respect to each of the color differ- 
ence signals U and V. the LPF 14a and 14b respectively perform low pass filtering to limit the band. Then, the thinning 
circuit 15 performs thinning processing which complies with a predetermined compression method. e.g., from Y:U:V = 
4:4:4 to Y:U:V = 4:2:2 or Y:U:V = 4:1:1. Then, a JPEG compression circuit 16 performs compression according to JPEG. 
[0065] Rg. 7 is a block diagram showing a configuration of the pseudo color removing processor 21 according to 
the fifth entixxiiment 

[0066] Assume that image data is generated in R. G and B planes by the color interpolator 9. The RGB/XYZ con- 
verter 21a converts image data from RGB color space to XYZ color space. The conversion is executed by the fbllowina 
equations (1 3) to (1 5) : 



X= 2.7689 X R + 1.7517 x G + 1.1302 x B (13) 

20 Y= 1 .0000 X R + 4.5907 x G + 0.0601 x B (14) 

Z= 0.0000 X R + 0.0565 x G + 5.5943 x B (15) 

[0067] Then. XYZ/L*a*b* converter 21b converts the image data from XYZ color space to L*a*b* color space. The 
25 conversion is executed by the following equations (1 6) to (1 8): 

L*= 1 16 (y/y o) -16 (Y/Yo>0.008856) (16) 
a*- 500 [(X^o) ' CY-Y^,) ^^] (17, 

30 ^ ' 

b*= 200 [(Y/Yo) • (Z-Zo) (18) 
[0068] Xo. Yo suxJ Zo represent tristimulus values of the perfect diffuse surface. 

[0069] Herein, median filters 21c and 21d, each sending as an isolated pointfilter. execute median filtering (median 
35 value filtering) respectively to a* and b* of the L*a*b*. 

[0070J Since the principle of the processing performed by the median filters 21 c and 21d is the same as the afore- 
mentioned description shown in Figs. 2A and 2B, description thereof will not be provided herein. 
[0071] Then. L*a*b*/XYZ converter 21 e converfe image data from L*a*b* color space to XYZ color space. Rnally. 
XYZ/RGB converter 21 f converts ttie image data from XYZ color space to RGB color space. By this, median filtering 
can be executed only in a color space that gives small influence to the luminance (resolution). Therefore, the decrease 
in resolution of an image can be prevented. 

[Sixtti Eftibodiment] 



[0072] Fig. 8 is a block diagram showing a configuration according to the sixtfi embodiment of the present invention. 
[0073] In the foregoing fifth embodiment, image data in RGB cotor space, which has been ouiputted by the pseudo 
color removing processor 21. is converted from RGB color space to YUV color space by the RGBr/UV converter 12 to 
be subjected to JPEG conTpression. This YUV conversion is executed by the aforementioned equations (1) to (3). 
[0074] According to the sixtti embodiment, instead of tine pseudo color removing processor 21 . ttie median filters 
(median value filters) 17b and 1 7c execute filtering to U and V signals outputted by the RGB/YUV converter 12 Then 
to perform JPEG compression, the LPF 1 4a and 1 4b perform low pass filtering to color difference data to limit the band 
of tile color difference signals. Then, the ttiinning circuit 15 performs thinning into a predetermined format, e.g., from 
Y:U:V = 4:4:4 to Y:U:V = 4:2:2 or Y:U:V = 4:1 :1. and compression Is performed by ttie JPEG compression circuit 16. 
[0075] For ttie isolated point removing filter, filtering called smoothing (Edge Preserving Smoothing) for preserving 
edges, which has been described in ttie fourtti embodiment, may be enployed besides ttie above-described median 
filter 

[0076] Furthermore, for tfie isolated point removing fnter. smoothing (Maximum Homogeneity Smoottiing) for pre- 
serving edges, which has been described in the fourth embodiment, may be employed. 
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[00771 Besides the above, filtering called hysteresis smoothing may be employed. More specifically, a hysteresis 
characteristic is produced for a data value, and noise is absorbed by the hysteresis characteristic. Any of the aforemen- 
tioned filters may be used as long as interpolation or smoothing is performed by using outputs of peripheral pixels of an 
isolated point having a pixel value largely different from peripheral pixel values. 

5 

[Seventh Embodiment] 

[0078] A characteristic feature of the seventh embodiment is described before explaining a specific example of the 
seventh ennbodiment. To eliminate the eibove-described pseudo colors, it is ideal to emplcy three-plate camera where 

w CCDs are provided for each of the R, G and B color components. With such structure, the size of a camera increases. 
In the seventh embodiment, an attention is directed to the fact that pseudo colors generated from red (R) to yellow (Y) 
and from blue (B) to cyan (0) have high frequencies. Thus, the characteristic feature of the seventh embodiment is to 
suppress generation of pseudo colors by reducing the gain of a color difference signal when the luminance signal has 
a high frequency component and the hue falls within a predetermined range. 

IS [0079] Fig. 9 is a block diagram showving, mainly, a construction of an image processing unit of a digital camera 
according to the seventh eiiibodiment of the present invention. 

[0080] Light 1 , incident upon the camera, passes through the lens 2. then the amount of the light is adjusted by the 
diaphragm 3. and the image sensing device 5 (hereinafter referred to as a CCD), e.g.. CCD or CMOS, is exposed while 
a shutter (not shown) is open. Before the light 1 is inddent upon the CCD 5. a frequency area (band) of a long wave- 

20 length is cut off by the IR fBter 4 so that the CCD 5 does not detect light in the infrared region. The light inddent upon 
the CCD 5 IS accumulated in the CCD 5 as an electric charge, and outputted as an electric signal. The electric signal 
is anplif ied by a predetermined gain by the CDS • AGC 6, and converted to digital data by the A/D converter 7. Image 
data converted to digital data in the foregoing manner is transferred to the white bateince drcuit 8 where R. G and B 
gains are adjusted. Then, in the color interpolator 9, the image data is generated in. for instance, three color (R, Q and 

25 B) planes as shown in Fig 10. The image data expressed in the R. G and B planes are then transferred to the masking 
processor 10 where adjustment is made for the hue of the R. G and B colors. Then, the gamma converter 11 performs 
necessary processing for displaying an image on a display or the like. Further, compression processing according to 
JPEG or the like is performed on the image data since the data in the R, G and B color planes has a large amount of 
data. 

30 [0081 1 Rrst. RGB/Y converter 22 converts RGB cotor space to color space of Y color difference signal. The conver- 
sion is executed by. for instance, above-inentioned equations (1) to (3). 

[0082] To detect high frequency components and hue. the image data, expressed in the R. G and B planes by the 
color interpolator 9, is subjected to calculation of a luminance signal Y by the RGB/Y converter 22, using the aforemen- 
tioned equation (1). Next, to obtain a high frequency component of the obtained luminance signal Y, a spatial high pass 

35 filter (HPF) 23 executes spatial high pass filter confutation as shown in Rg. 1 1 . The luminance signal Y. obtained by 
the RGB/Y converter 22. is transferred to a hue detection circuit 24 for detecting a hue, where color difference signals 
R-Y and B-Y are calculated and a hue is detected based on the calculated results. The detected hue data is outputted 
to a pseudo color suppression circuit 13, Referring to Rg. 1 1 . spatial high pass filter corrputation is executed by multi- 
plying an input signal by a matrix 300, and subtracting the input signal from the multiplication resuH by a subtracter 301 . 

40 [0083] The pseudo color suppression circuit 13 suppresses pseudo colors by reducing gains of U and V. outputted 
by the RGB/Y converter 22. based on the high frequency component of the luminance signal Y obtained by the spatial 
high pass filter 23 arxi the hue data obtained by the hue detection circuit 24. 
[0084] The pseudo color suppression processing is described with reference to Rgs. 12 and 13. 
[0085] First, it is determined whether or not a color of the pixel of interest falls within a predetermined range (area 

45 indicated by hatching in Fig. 13) in Y R-Y. B-Y color space. If the color is determined to be in the range where it is con- 
sidered as a pseudo color, gains of the color difference signals U and V. having a predetermine value of1 or larger as 
shown in Fig. 12. are reduced in accordance with a high frequency component value of the luminance signal outputted 
by the spatial high pass filter 23. By this, pseudo colors can be suppressed. 

[0086] Pseudo color suppression processing performed in the above<Jescribed configuration is explained. A high 
so frequency component, such as that having a vertical stripe pattern or a checker flag pattern described with reference 
to Rgs. 19 to 22, is extracted Ijy the spatial HPF 23. The aforementioned pseudo color generation is prevented by 
reducing gains of color difference signals U and V in the extracted high frequency area, which m£^ cause pseudo colors 
described in Rgs. 19 to 22 (for instance, predetermined areas of R and Y components in Rg. 19. predetermined areas 
of C and B conponents in Rg. 20, or predetermined areas of R and B components in Rgs. 21 and 22, all of which cor- 
. 55 respond to the hatching portion in Fig. 13). 

[0087] Next, the low pass filter (LPF) 14 execute low pass filtering in the horizontal or vertical direction. Then, the 
thinning circuit 15 performs thinning processing of the color difference signals, in accordance with a predetermined 
JPEG format e.g., from Y:U:V = 4:4:4 to Y:U:V = 4:2:2 or Y:U:V = 4:1:1. Then, the JPEG compression drcuit 16 per- 
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forms JPEG compression. By this, a compressed image having less pseudo colors can be generated. 
[00881 Note herein that instead of calculating the luminance signal Y by the aforementioned equation (1). the cal- 
culation may be executed with a simplified luminance signal Ye. For instance, aforementioned equation (7) 
Ye=R + 2xG + B. may be used. 

[0089] Alternatively, a green (G) signal component may be used as a luminance signal without calculatina the lumi- 
nance signal. 

[0090] Rg. 14 is a flowchart explaining pseudo color removing processing accoirling to the seventh embodiment of 
the present invention. 

[00911 Refemng to Fig. 14. in step SI R. G and B signals of an image, upon which color interpolation processing 
has been performed, are inputted and a luminance signal Y is generated from the R. G and B signals by the RGBA^ 
converter 22. In step S2. high frequency components of the luminance signal Y are extracted by the spatial high pass 
filter(HPF) 23. In step S3, differences are respectively calculated between each of R. B signals and the luminance sig- 
nal Y by the hue detection circuit 24. and color difference signals R-Y and B-Y are generated. In step S4. a hue signal 
IS generated based on the color difference signals, and it is determined whether or not the hue signal falls within a color 
range that is considered as pseudo colors. If it is determined as a pseudo color in step S5. the control proceeds to step 
S6 where gains off the color difference signals in the pseudo color portion are reduced in accordance with the amount 
of high frequency component of the pixel. By this, for instance, in the color space corresponding to the hatching area in 
Rg, 13. pseudo color components generated by color interpolation can be suppressed. Then the process proceeds to 
step S7, isolated pixels of the image are removed. 

[00921 As described above, the seventh embodiment achieves an effect of preventing pseudo colors generation 
from red to yellow and from blue to cyan. 

[Eighth Embodiment] 

[00931 A characteristic feature of the eighth embodiment is described before explaining a specific example of the 
eighth embodiment. To eliminate the above-described pseudo colors, it is ideal to employ three-plate camera where 
CCDs are provided for each of the R. G and B color components. With such structure, the size of a camera increases 
In the eighth embodiment, an attention is directed to the fact that pseudo colors generated from red (R) to yellow (Y) 
and from blue (B) to cyan (C) have high frequencies. Thus, the characteristic feature of the eighth embodiment is to sup- 
press generation of pseudo colors by removing an isolated point of a luminance signal, having a high frequency com- 
ponent and. if the hue falls within a predetermined range, reducing the gain of the color difference signal. 
[00941 Rg. 15 is a block diagram showing, mainly, a construction of an image processing unit off a digital camera 
according to the eighth embodiment of the present invention. 

[0095] The pseudo color suppression circuit 1 3 suppresses pseudo colors by redudng gains of U and V, outputted 
by the RGB/YUV converter 22. based on a high frequency component of the luminance signal Y obtained by the spatial 
hjgh pass filter 23 and hue data obtained by tiie hue detection circuit 24. The pseudo color suppression processing is 
perfomied in the same manner as that of the afc>ove<lescribed seventh embodiment. 

[0096] Among the above-described pseudo colors, patterns of pseudo cotors from red (R) to yellow (Y) and from 
blue (B) to cyan (C) can be determined as pseudo colors by detecting high frequency conponents off the luminance sig- 
nal because the degree of luminance of pseudo colors are different from others. However, pseudo colors generated 
around a character are often isolated, and tiie degree of luminance of ht pseudo colors are not alwmrs different from the 
peripheral pixel luminance. 

[0097] In view of tiie above, according to the eighth embodiment the median filters 24a and 24b remove isolated 
points with respect to the color difference signals U and V and further perform pseudo color removal. Rgs 2A and 2B 
explain this processing. Herein, elements having 3x3 pixel area including a pixel of interest are arranged in the ascend- 
ing order, and the value of the pixel of interest is replaced witfi a median value of the arranged pixels. By this a pixel 
having a color of extremely different hue ffrom the hues of peripheral pixels, can be eliminated. Particularly since the 
low pass ffiltenng is not executed In tiie eighth embodiment, resolution does not decrease largely, thus does not influ- 
ence peripheral pixels. 

[0098] Pseudo color suppression processing performed in the above-described configuration is explained A high 
frequency component, such as that having a vertical stipe pattern or a checker flag pattern described witti reference 
to Figs. 19 to 22. is exti-acled by the spatial HPF 23. The aforementioned pseudo color generation is prevented by 
reducing gams of color difference signals U and V in the extracted high frequency area, which may cause ttie pseudo 
colors descnbed in Rgs. 19 to 22 (for instance, predetermined areas of R and Y components in Rg. 19. predetermined 
areas of C and B components in Fig. 20. or predetermined areas of R and B components in Figs. 21 and 22. all of which 
corre^ond to the hatching portion in Rg. 13). After pseudo colors are suppressed in the foregoing manner, tiie median 
filters 24a and 24b execute above-described filtering, thereby removing isolated pixels. 

[0099] Next, the low pass filter (LPF) 14a and 14b execute low pass filtering in the horizontal or vertical direction 
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Then, the thinning circuit 15 performs thinning processing of the color difference signals, in accordance with a prede- 
termined JPEG format, e.g.. from Y:U:V = 4:4:4 to Y:U:V = 45:2 or Y:U:V = 4:1 :1 . Then, the JPEG compression circuit 
16 performs JPEG compression. By this, a compressed Image having less pseudo colors can be generated. 
[0100] Note that, to calculate the luminance signal Y, the aforementioned equation (7) may be used. 
5 [01 01 ] Alternatively, a green (Q) signal component may t>e used as a luminance signal without calculating the lumi- 
nance signal. 

[01021 Since the pseudo color removal processing according to the eighth embodiment is performed in the same 
manner as that explained in the flowchart in Fig. 14. detailed description will be omitted herein. 

10 [Ninth Embodiment] 

[0103] In the foregoing seventh and eighth emtxxJiments, means for detecting pseudo colors is provided after the 
color interpolator 9 and pseudo cofor detection is performed using signals in the R, G and B planes. The reason is that, 
since processing such as masking processing, y conversion arxi so forth that may cause changes in hue are performed 

IS after the color interpolator 9. the hue range of the detected pseudo colors may change. Because of this reason, the 
luminance signal Y is calculated to obtain color difference signals R-Y and B-Y. However, the pseudo color detection 
means does not have to be provided in this position. For instance, in order to perform JPEG compression. RGB color 
space must be converted to a color space of luminance or color difference. Therefore, pseudo colors can be detected 
from these signals. The ninth embodiment relates to such casa Note that, as mentioned above, there is a possibility 

20 that the hue range of pseudo colors changes depending on images. 

[01 04] Fig. 1 6 is a block diagram showing a construction of an image processing unit of a digital camera according 
to the ninth embodiment of the present invention. For the components that are comnrK)n to the construction shown in 
Fig. 9. the same reference numerals are assigned. 

[01 05] Light 1 . incident upon the camera, passes through the lens 2. then the amount of the light is adjusted by the 
25 diaphragm 3. and the CCD 5 is exposed while a shutter (not shown) is open. Before the light 1 is incident upon the CCD 
5. a frequency area of a long wavelength is cut off by the IR filter 4 so that the CCD 5 does not detect light in the infrared 
region. The light incident upon the CCD 5 is accumulated in the CCD 5 as an electric charge, and oulputted as an elec- 
tric signal. The electric signal is amplified by a predetermined gain by the CDS * AGC 6. and converted to digital data 
by the A/D converter 7. Image data converted to digital data In the foregoing manner is transferred to the white balarx:e 
30 circuit 8 where R. G and B gains are adjusted. Then, in the color interpolator 9. the image data is generated in, for 
instance, three color (R. G and B) planes as shown in Rg. 2. The image data expressed in the R. G and B planes are 
then transferred to the masking processor 1 0 where adjustment is made for the hue of the R. G and B cofors. Then, the 
gamma converter 11 performs necessary processing for displaying an image on a display or the like. Then, RGBA^UV 
converter 12 converts the image data from RGB cofor space to color space of Y color difference signal. 
35 [0106] Among the converted signals, luminance signal Y Is transmitted to the spatial high pass filter (HPF) 25 for 
detecting high frequency components of the signal by the filtering shown in Fig. 11. Then, a hue detection circuit 26 
determines whether or not a pixel of Interest ^lls within the hue range of pseudo colors. Herein, instead of the above- 
described color space R-Y and B-Y shown in Fig. 13. U and V outputted by the RGB/YUV converter 12 are used as 
shown in Fig. 17. 

40 [01 07] As similar to the seventh and eighth embodiments, if it is determined that a color of the pixel of interest falls 
within the hue range of pseudo colors, the pseudo color suppression circuit 13 reduces gains of the color difference sig- 
nals U arKi V. having a value of 1 or larger as shown in Rg. 1 2. in accordance with an output value of the spatial HPF 25. 
[01 08] In this case, for instance, it may be necessary to change, for instance, the hue range depending on the white 
balance. f=br instance, when the gain of red is increased, the hue range is shifted towards red. or a hue angle is wid- 

45 ened. 

[0109] Next, the LPF 14aand 14b execute low pass filtering in the horizontal or vertical direction. Then, the thinning 
circuit 15 performs thinning processing of the color difference signals, in accordance with a predetermined JPEG for- 
mat, e g., from Y:U:V = 4:4:4 to Y:U:V = 4:2:2 or Y:U:V = 4:1 :1 . Then, the JPEG compression circuit 16 performs JPEG 
compression. 

so [01 10] By this, a compressed image having less pseudo colors can be generated. 
[Tenth Embodiment] 

[Oil 1] In the foregoing seventh and eighth embodiments, means for detecting pseudo colors is provided after the 
55 color interpolator 9 and pseudo color detection is performed using signals in the R. G and B planes. The reason is that, 
since processing such as masking processing, y conversion and so forth that may cause changes in hue are performed 
after the color interpolator 9. the hue range of the detected pseudo colors may change. Because of this reason, tiie 
luminance signal Y is calculated to obtain color difference signals R-Y and B-Y. However, the pseudo color detection 
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means does not have to be provided in this position. For instance, in order to perform JPEG compression, RGB color 
space must be converted to a color space of luminance and color difference. Therefore, pseudo colors can be detected 
from these signals. With reference to the foregoing eighth embodiment (Fig. 15), in the median filter 24a and 24b 
where pseudo color suppression is performed, delay is generated between the luminance signal Y and color difference 
5 signals U and V. Because of this, the luminance signal must also be delayed. The tenth embodiment makes use of this 
delay. 

[01 1 21 Rg. 1 8 is a block diagram showing a construction of an image processing unit of a digital camera according 
to the tenth embodiment of the present invention. For the components that are common to the construction shown in 
Fig. 15. the same reference numerals are assigned. 

10 [Oil 31 Ught 1 , incident upon the camera, passes through the lens 2, then the amount of the light is adjusted by the 
diaphragm 3, and the CCD 5 is exposed while a shutter (not shown) is open. Before the light 1 is incident upon the CCD 
5, a frequency area of a long wavelength is cut off by the IR filter 4 so that the CCD 5 does not detect light in the infrared 
region. The light incident upon the CCD 5 is accumulated in the CCD 5 as an electric charge, and outputted as an elec- 
tric signal. The electric signal is amplified by a predetermined gain by the CDS • AGC 6, and converted to digital data 

15 by the A/D converter 7. Image data converted to digital data in the foregoing manner is transferred to the white balance 
circuit 8 where R. G and B gains are adjusted. Then, in the color interpolator 9. the image data is generated in. for 
instance, three color (R, G and B) planes as shown in Rg. 2, The image data expressed in the R. G and B planes are 
then transfenred to the masking processor 10 where adjustment is made for the hue of the R. G and B colors. Then, the 
gamma converter 1 1 performs necessary processing for displaying an image on a display or the like. Then. RGB/YUV 

20 converter 1 2 converts the image data from RGB color space to color space of Y color difference signal. 

[01 14] Among the converted signals, luminance signal Y is transmitted to the spatial high pass filter (HPF) 25 for 
detecting high frequency conponents of the luminance signal by the filtering shown in Fig. 1 1 . Meanwhile, color differ- 
ence signals U and V are transmitted to the median filters 24a and 24b for executing the above-described median filter- 
ing shown in Rgs. 2A and 2B. By virtue of this, while high frequency components of the luminance signal Y are 

25 detected, median filtering is executed to the color difference signals. Therefore, it is possible to reduce wastefol use of 
a memory caused by delay of one of the signals. 

[0115] Next, the hue detection circuit 26 determines whether or not a pixel of interest falls within the hue range of 
pseudo colors. Herein, instead of the above-described color space R-Y and B-Y shown in Fig. 13. U and V outputted 
by the RGBA^UV converter 12 are used as shown in Rg. 17. 

30 [01 16] As similar to the eighth embodiment if it is determined that a color of the pixel of interest ^Is within the hue 
range of pseudo colors, the pseudo color suppression circuit 13 reduces gains of the color difference signals U and V. 
having a value ofl or larger as shown in Fig. 12, in accordance with an output value of the spatial HPF 25. 
[01 1 7J In this case, for instance, it may be necessary to change, for instance, the hue range depending on the white 
balance. For instance, when the gain of red is increased, the hue range is shifted to the red. or a hue angle is widened. 

35 [01 18] Next, the LPF 1 4a and 1 4b execute low pass filtering in the horizontal or vertical direction. Then, the thinning 
circuit 15 performs thinning processing of the color difference signals, in accordance with a predetermined JPEG for- 
mat. e.g.. from Y:U:V = 4:4:4 to Y:U:V = 42:2 or Y:U:V = 4:1 :1 . Then, the JPEG compression circuit 16 pertbrms JPEG 
compression. 

[01 19] By this, a compressed image having less pseudo colors can be generated. 

40 [01 20] The present invention can be applied to a system constituted by a plurality of devices (e.g., host coirputer. 
interface, reader, printer) or to an apparatus comprising a single device (e.g.. copying machine, facsimile machine). 
[0121] Further, the object of the present invention can also be achieved by providing a storage medium storing pro- 
gram codes for performing the aforesaid processes to a computer system or apparatus (e.g., a personal computer) 
reading the program codes, by a computer (CPU or MPU) off the system or apparatus, from the storage medium, then 

45 executing the program. 

[01 22] In this case, the program codes read from the storage medium realize the functions according to the embod- 
iments, and the storage medium storing the program codes constitutes the invention. 

[01 23] Further, ttie storage medium, such as a floppy disK a hard disk, an optical disk, a magneto-optical disk. CD- 
ROM. CD-R. a magnetic tape, a non-volatile type memory card, and ROM can be used for providing the program codes. 
50 [01 24] Furthermore, besides aforesaid fonctfons according to the above embodiments are realized by executing the 
program codes which are read by a computer, the present invention includes a case where an OS (operating system) 
or the like working on the computer performs a part or ttie entire processes in accordance witti designations of the pro- 
gram codes and realizes functions according to tiie above embodiments. 

[0125] Furthermore, the present invention also includes a case where, after the program codes read from the stor- 
55 age medium are written in a function expansfon card which is inserted into the computer or in a memory provided in a 
function expansion unit which is connected to the computer. CPU or the lite contained in the function expansion caid 
or unit pertbrms a part or the entire process in accordance witti designations of the program codes and realizes func- 
tions of the above embodiments- 
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[0126] As has been set forth above, according to the above-described embodiments, RGB color space is converted 
to YU V or Y R-Y. B-Y or G. R-G. B-G color space, and isolated point removing filtering (e.g., median filter or the like) is 
executed respectively to color difference signals in UV. or R-Y. B-Y or R-G, B-G color space, thereby enat}ling reduction 
of pseudo colors generated by color interpolation. 

5 [0127] Furthermore, according to the above-describe embodiments, even if an optical low pass filter is not provided 
before the image sensing device, such as CCD or the like, pseudo color components can be reduced by converting 
RGB color space to L*a*b* or YUV color space, then performing filtering to color difference signals such as a* and 
signals or U and V signals with the isolated point removing filter, and then performing compression according to JPEG 
or the like. Accordingly, an image, having less t^ock distortion and reduced pseudo color conrponents, can be obtained. 

10 [0128] Furthermore, according to the atx^ve-described embodiments, pseudo colors generated by color interpola- 
tion can be suppressed by detecting high frequency components of a luminance signal arxJ pseudo cobrs generated 
from red to yellow and from blue to cyan in the hue. and then reducing gains of color difference signals U and V. 
[01 29] Still further, according to the above-described emtxxliments. pseudo colors generated by color Interpolation 
can be suppressed by detecting high frequency components of a luminance signal and pseudo colors generated from 

IS red to yellow and from blue to cyan in the hue, then reducing gains of color difference signals U and V. and further elim- 
inating isolated pixels. 

[0130] Still further, according to the above-described embodiments, pseudo colors generated by color interpolation 
can be suppressed by executing median filtering to color difference signals in the color space of luminance and color 
difference signals, then detecting high frequency components of a luminance signal and pseudo colors generated from 
20 red to yellow and from blue to cyan in color difference sigrials. and reducing gains of color difference signals U and V 
[01 31 ] Still further, according to the atxive-described embodiments, by executing median filtering to color difference 
signals while detecting high frequency components of a lunninance signal, it is possit)le to eliminate a processing circuit 
necessary to match delayed timing of the luminance signal and color difference signals. 

[0132] The present invention is not limited to the above embodiments and various changes arKi modifications can 
25 be made within the spirit and scope of the present invention. Therefore, to apprise the put^lic of the scope of the present 
invention, the following claims are made. 

Claims 

30 1 . An image sensing apparatus characterised by comprising: 
an image sensing device; 

focusing means for focusing into an image on said image sensing device: 

A/D conversion means for converting an image signal outputted by said image sensing device into a digital sig- 
35 nal; 

color interpolation means for performing color interpolation on the cfigital signal converted by said A/D conver- 
sion means and generating image data on a plurality of color planes; 

color space conversion means for converting a color space of the plurality of color planes to a color space of 
another colorimetric system; and 
40 pseudo color removing means for reducing a color component, generated by said color interpolatbn means, 

by controlling a color difference signal converted by said color space conversion means. 

2. The image sensing apparatus according to claim 1, characterised in that said pseudo color removing means 
includes an isolated poirrt removing filter which replaces a value of a pixel of interest with a substantial median pixel 

45 value of peripheral pixels of the pixel of interest. 

3. The image sensing apparatus according to daim 2. characterised in that said isolated point removing filter includes 
a median value filter or median filter. 

50 4. The image sensing apparatus according to claim 1 . characterised in that said color interpolation means generates 
image data in R. G and B planes. 

5. The image sensing apparatus according to claim 1 or 4. characterised in that said color space conversion means 
converts RGB color space to YUV color space. 

55 

6. The image sensing apparatus according to claim 1 or 4. characterised in that said color space conversion means 
converts RGB color space to Y. R-Y, B-Y color space. 
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7. The image sensing apparatus according to claim 1 or 4. characterised in that said color space conversion means 
converts RGB color space to G, R-G. B-G color space. 

8. The image sensing apparatus according to any one of claims 1-7. characterised in that said focusing means 
includes an infrared ray filter, or an infrared ray filter and optical low pass filter. 

9. An image sensing apparatus characterised by comprising: 
an image sensing device; 

focusing means for focusing into an image on said image sensing device; 

A/D conversion means for converting an image signal outputted by said image sensing device into a digital sig- 
nal; ^ 

color interpolation means for performing color interpolation on the digital signal converted by said AJD conver- 
sion means and generating image data on a plurality of color planes; 

pseudo color removing means for reducing a color component included in image data, on which color interpo- 
lation is performed by said color interpolation means; 
color space conversion means for converting a color space of the plurality of color planes, where the color com- 
ponent IS reduced by said pseudo color removing means, to a color space of another colorimetric system and 
compression means for compressufig image data where color space is converted by said color space oonver- 
20 sion means. 

10. The image sensing apparatus according to daim 9. characterised in that said pseudo color removing means com- 
prises: 



10 



15 



25 



30 



first color space conversion means for converting a color space of the plurality of color planes to a color space 
of arxjther colorimetric system; 

isolated point removing means for reducing an isolated pseudo color component by controlling signals except 
a luminance component signal of the colorimetric system converted by said first color space conversion 
means; and 

second color space conversion means for converting the color space, converted by said first color space con- 
version means, to an original color space. 



11. The image sensing apparatus according to claim 10. characterised in that said isolated point removing means 
includes an isolated point removing filter which replaces a value of a pixel of interest with a substantial median pixel 

35 value of peripheral pixels of the pixel of interest. 

12. The image sensing apparatus according to claim 11, characterised in that said isolated point removing filter 
includes a median value filter or median filter. 

40 13. The image sensing apparatus according to daim 10, characterised in that said color interpolation means generates 
image data in R, G and B planes. 



14. The image sensing apparatus according to claim 1 1 , characterised in that said first color space 



conversion means 



converts RGB color space to L*a*b* color space, and said isolated point removing means causes the isolated point 
45 removing filter to filter a frequency range of a* and b* signals. 

15. The image sensing apparatus according to any one of claims 10-14. characterised in that said color space conver- 
sion means converts RGB color space to YUV color space. 

so 16. An image sensing apparatus characterised by comprising: 

an image sensing device; 

focusing means for focusing into an image on said image sensing device; 

A/D conversion means for converting an image signal outputted by said inWge sensing device info a digital sig- 
55 nal; 

color interpolation means for performing color interpolation on the digital signal converted by said A/D conver- 
sion means and generating image data on a plurality of color planes; 

separation means for separating the image data in the plurality of ccilor planes into luminance data and color 
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difference data; 

extraction means for extracting a high frequency component from the luminance data separated by said sepa- 
ration means; and 

pseudo color removing means for reducing a color component generated by said color interpolation means. In 
5 accordance with the high frequency component of the luminance data extracted by said extraction means and 

hue data obtained from the color difference data. 

17. The image sensing apparatus according to daim 16. characterised in that said pseudo color removing means com* 
prises: 

10 

determination means for determining whether or not the high frequency component of the luminance data and 
the hue data fall within a color range of the pseudo color; arxi 

means for reducing a value of the color difference data, determined to be within the color range by said deter> 
mination means. 

IS 

18. An image sensing apparatus characterised by comprising: 

' an image sensing device; 
focusing means for focusing into an image on said image sensing device; 
' 20 A/D conversion means for converting an image signal outputted by said image sensing device into a digital sig- 

nal; 

color interpolation means for performing color interpolation on the digital signal converted by said A/D conver- ^. . 

sion means and generating image data on a plurality of color planes; 

color space conversion means for converting a color space of the plurality of color planes to a color space of 
25 another cdorimetric system; 

separation means for separating image data in the color space of another colorimetric system into luminance 
data and hue data; . 
extraction means for extracting a high frequency component from the luminance data separated by said sepa- 
ration means; and 

30 pseudo color removing means for reducing a color component generated by said color interpolation means, in 

accordance with the high frequency component of the luminance data ^racted by said extraction means and 
the hue data 

19. The image sensing apparatus according to claim 1 8. characterised in that said pseudo color removing means com- 
as prises: 

determination means for determining whether or not the high frequency component of the luminance data and 
the hue data fall within a color range of the pseudo color; and 

means for reducing a value of the color difference data, determined to be within the color range by said deter- 
( 40 mination means. 

20. The image sensing apparatus according to daim 1 8 or 1 9, characterised in that said color space conversion means 
converts the color space to YUV, or Y, R-Y, B-Y, or G, R-G. B-G color space. 

45 21 . The image sensing apparatus according to any one of claims 1 8-20» characterised by further comprising a low pass 
filter for removing a high frequency component from a color signal, from which a pseudo color is removed by said 
pseudo color removing means. 

22. The image sensing apparatus according to any one of claims 18-21. characterised in that the color range of a 
50 pseudo color indudes a color area from red to yellow, and a color area from blue to cyan. 

23. The image sensing apparatus according to any one of claims 18-22, characterised by further corrprising isolated 
point removing means for removing an isolated pixel from a hue signal, from which the pseudo color is removed by 
said pseudo color removing means. 

55 

24. The image sensing appeu-atus according to daim 23. charadertsed in that said isolated point removing means 
includes a median filter. 



15 



EP 1 022 912 A2 



15 



20 



25. An image sensing apparatus characterised by comprising: 

an image sensing device; 

focusing means for focusing into an image on said image sensing device; 
5 fijD conversion means for converting an image signal outputted by said image sensing device into a digital sig- 

nal; 

color interpolation means for performing color interpolation on the digital signal converted by said A/D conver- 
sion means and generating image data on a plurality of color planes; 

color space conversion means for converting a color space of the plurality of color planes to a color space of 
^0 another colorimetric system; 

separation means for separating image data in the color space of another colorimetric system into luminance 
data and hue data; 

extraction means for extracting a high frequency component from the luminance data separated by said sepa- 
ration means; 

isolated point removing means for removing an isolated pixel based on the hue data separated by said sepa- 
ration means; and 

pseudo color removing means for reducing a color component generated by said color interpolation means, in 
accordance with the high frequency component of the luminance data extracted by said extraction means and 
the hue data from which an isolated pixel is removed by said isolated point removing means. 

26. The image sensing apparatus according to claim 25. characterised in that said pseudo cotor removing means com- 
prises: 

determination means for determining whether or not the high frequency component of the luminance data and 
25 tiie hue data fall within a color range of the pseudo color component; and 

means for reducing a value of the color difference data, determined to be within the color range by said deter- 
mination means. 

27. The image sensing apparatus according to claim 25 or 26, characterised in that said color space conversion means 
30 converts the color space to YUV, or Y. R-Y, B Y. or G. R-G. B-G color spaca 

28. The image sensing apparatus according to any one of claims 25-27. characterised by further comprising a low pass 
filter for removing a high frequency component from a color signal, from which a pseudo color is removed by said 
pseudo color removing means. 

35 

29. The image sensing apparatus according to any one of claims 25-28. characterised in that the color range of the 
pseudo color includes a color area from red to yellow, and a color area from blue to cyan. 

30. An image processing metiiod for an image sensing apparatus which includes an image sensing device and gener- 
ates an image signal corresponding to an image formed on the image sensing device, characterised by comprising: 

an fiJO conversion step of converting an image signal outputted by the image sensing device into a digital sig- 
nal; 

a color interpolation step of performing color interpolation on the digital signal converted in said A/D conveision 
45 step and generating image data on a plurality of color planes; 

a color space conversion step of converting a color space of the plurality of color planes to a color space of 
another colorimetric system; and 

a pseudo color removing step of reducing a color component, generated in said color interpolation step, by 
controlling a cotor difference signal converted In said color space conversion step. 



40 



50 



31. The image processing method according to daim 30. characterised in that in said pseudo color removing step, a 
value of a pixel of interest is replaced with a substantial median pixel value of peripheral pixels of the pixel of inter- 
est 

55 32. The image processing method according to claim 30. characterised in that in said pseudo color removing step fil- 
tering is performed by a median value filter or median filter. 

33. The image processing method according to any one of claims 30-32. characterised in that in said color interpolation 
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step, image data is generated in R. G and B planes: 



34. The image processing method according to any one of claims 30-32. characterised in that in said color space con- 
version step, RGB color space is converted to YUV color space. 

5 

35. The image processing method according to any one of claims 30-32, characterised in that in said color space con- 
version step, RGB color space is converted to Y, R-Y. B-Y color space. 

36. The image processing method according to any one of daims 30-32. characterised in that in said color space con- 
10 version step. RGB color space is converted to G. R-G. B-G color space. 

37. An image processing method for an image sensing apparatus which includes an image sensing device and gener- 
ates an image signal corresponding to an image formed on the image sensing device, characterised by comprising: 

IS an A/D conversion step of converting an image signal outputted by the image sensing device irrto a digital sig- 

nal; 

a color interpolation step of performing color interpolation on the digital signal converted in said A/D conversion 
step and generating image data on a plurality of color planes: 

a pseudo color removing step of reducing a color component included in image data, on which color interpda- 
^' 20 tion is performed in said color int^polation step; 

a color space conversion step of converting a color space of the plurality of color planes, where a pseudo color 
component is reduced in said pseudo color removing step, to a color space of another colorimetric system; and 
a compression step of compressing image data where color space is converted in said color space conversion 
step. 

25 

38. The image processing method according to claim 37, characterised in that in said pseudo color removing step, a 
value of a pixel of interest is replaced with a suiDstantial median pixel value of peripheral pixels of the pixel of inter- 
est. 

30 39. The image processing metiiod according to claim 37. characterised in that in said pseudo color removing step, f fl- 
tering is performed by a median value f flter or median filter. 

40. The image processing method according to any one of claims 37-39, characterised in that in said color interpolation 
step, image data is generated in R. G and B planes. 

35 

41 . The image processing metiiod according to any one of claims 37-40, characterised in that in said color space con- 
version step. RGB color space is converted to L*a*b* color space, and filtering is performed to a* and b* signals in 
said pseudo color removing step. 

(' 40 42- The image processing niethod according to any one of claims 37-41 , characterised by further comprising a conver- 
sion step of converting the signal, processed in said pseudo color removing step, into YUV color space. 

43. The image processing method according to any one of claims 37-42, characterised in that in said compression 
step, image data converted to YUV color space is compressed. 

45 

44. An image processing method for an image sensing apparatus which includes an image sensing device and gener- 
ates an image signal corresponding to an image formed on the image sensing device, characterised tyy comprising: 

an A/D conversion step of converting an image signal outputted by the image sensing device into a digital stg- 
50 nal; 

a color interpolation step of performing color interpolation on the digital signal converted in said fiJD conversion 
step and generating image data on a plurality of color planes; 

a separation step of separating the image data in the plurality of color planes into luminance data and color dif- 
ference data; 

55 an extraction step of extracting a high frequency component from the luminance data separated in said sepa- 

ration step; arxJ 

a pseudo color removing step of reducing a color component generated in said color interpolation step, in 
accordance with the high frequency component of the luminance data extracted in said extraction step and hue 
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data obtained from the color difference data. 



45. The image processing method according to claim 44, characterised in that said pseudo color removing step com- 
prises: 

5 

a determination step of determining whether or not the high frequency component of the luminance data and 
the hue data fall within a color range of the pseudo color; and 

a step of reducing a value of the color difference data, determined to be within the color range in said determi- 
nation step. 

10 

46. The image processing method according to claim 44 or 45. characterised by further comprising a step of removing 
a high frequency component from a color signal, from which the pseudo color is removed in said pseudo color 
removing step. 

IS 47. The image processing method according to any one of claims 44-46, characterised in that the color range of a 
pseudo color includes a color area from red to yellow, and a color area from blue to cyan. 

48. An image processing method for an image sensing apparatus which includes an image sensing device and gener- 
ates an image signal corresponding to an image formed on the image sensing device, characterised by comprising: 

20 

an NO conversion step of converting an image signal oulputted by the image sensing device into a digital sig- 
nal; 

a color interpolation step of performing color interpolation on the digital signal converted in said A/D conversion 
step and generating image data on a plurality of color planes; 

a color space conversion step of converting a color space of the plurality of color planes to a color space of 
another colorimetric system; 

a separation step of separating image data in the color space of another colorimetric system into luminance 
data and hue data; 

an extraction step of extracting a high frequency component from the luminance data separated in said sepa- 
30 ration step; and 

a pseudo color removing step of reducing a color component generated in said color interpolation step, in 
accordance with the high frequency component of the luminance data extracted in said extraction step and the 
hue data. 

35 49. The image processing method according to claim 48. characterised in ttiat said pseudo color removing step com- 
prises: 

a determination step of determining whether or not the high frequency corrponent of the luminance data and 
the hue data fall within a color range of the pseudo color; and 

a step of reducing a value of the color difference data, determined to be witiiin the color range by said deter- ( 
mination step. 

50. The image processing method according to claim 48 or 49. characterised in that in said color space conversion 
step, ttie color space is converted to YUV, or Y, R-Y, B-Y. or G. R-G. B-G color space. 

45 

51 . The image processing method according to any one of claims 48-50. characterised by further comprising a step of 
removing a high frequency component from a color signal, from which the pseudo color is removed in said pseudo 
color removing step. 

50 52. The image processing step according to any one of claims 48-51 . characterised in that the color range of a pseudo 
color component includes a color area from red to yellow, and a color area from blue to cyan. 

53. An image processing method for an image sensing apparatus which includes an image sensing device and gener- 
ates an image signal corresponding to an image formed on the image sensing device, characterised by comprising: 

55 

an A/D conversion step off converting an image signal outputted by the image sensing device into a digital sig- 
nal; 

a color interpolation step of performing color interpolation on the digital signal converted in said A/D conversion 
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step and generating image data on a plurality of color planes; 

a separation step of separating image data in the plurality of color planes into luminance data and color differ- 
ence data; 

an extraction step of extracting a high frequency component from the luminance data separated in said sepa- 
ration step; 

a pseudo color removing step of reducing a color component generated in said color interpolation step, in 
accordance with the high frequency component of the luminance data extracted in said extraction step and the 
hue data obtained from the color difference data; and 

an isolated point removing step of renx)ving an isolated pixel from the hue data, from which a pseudo color 
component is removed in said pseudo color removing step. 

54. The image processing method according to claim 53. characterised in that said pseudo color removing step com- 
prises: 

a determination step of determining whether or not the high frequency component of the luminance data and 
the hue data fall within a color range of a pseudo color component; and 

a step of reducing a value of the color difference data, determined to be within the color range in said determi- 
nation step. 

55. The image processing method according to claim 53 or 54, characterised by further comprising a step of removing 
a high frequency component from a color signal, from which the pseudo color is removed in said pseudo color 
removing step. 

56. The image processing method according to any one of claims 53-55. characterised in that the color range of the 
pseudo color conponent includes a color area from red to yellow, and a color area from blue to cyan. 

57. An image processing method for an image sensing apparatus which includes an image sensing device and gener- 
ates an image signal corresponding to an image formed on the image sensing device, characterised by comprising: 

an A/D conversion step of converting an image signal oulputted by the image sensing device into a digital sig- 
nal; 

a color interpolation step of performing color interpolation on the digital signal converted in said A/D conversion 
step and generating image data on a plurality of color planes; 

a color space conversion step of converting a color space off the plurality of cotor planes to a color space of 
another colorimetric system; 

a separation step of separating image data in the color space off another cobrimetric system into luminance 
data and hue data; 

an isolating point removing step off removing an isolated pixel based on the hue separated in said separation 
step; 

an extraction step of extracting a high frequency component from the luminance data separated in said sepa- 
ration step; and 

a pseudo color removing step of reducing a color component generated in said color interpolation step, in 
accordance with the high frequency component of the luminance data extracted in said extraction step arKJ the 
hue data from which an isolated pixel is removed in said isolated point removing step. 

58. The Image processing method according to claim 57, characterised in that said pseudo color removing step com- 
prises: 

a determination step of determining whether or not the high frequency component of the luminance data and 
the hue data fall within a color range of the pseudo color; and 

a step of reducing a value of the color diffference data, determined to be within tiie color range in said determi- 
nation step. 

59- The image processing method according to claim 57 or 58, characterised in that in said color space conversion 
step, tiie color space is converted to YUV. or Y. R-Y, B-Y, or G. R-G. B-G color space. 

60. The image processing method according to any one of claims 57-59. characterised by further comprising a step of 
removing a high frequency component from a color signal, from which the pseudo color is removed in said pseudo 
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color removing step. 

61- The image processing method according to any one of claims 57-60, characterised in that the color range of the 
pseudo color component includes a color area from red to yellow, and a color area from blue to cyan. 

5 

62. A computer readable recording medium for storing a program for implementing the method according to any one 
of claims 30-61 . 
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